Method and apparatus for facilitating routing protocol redundancy in a network element

ABSTRACT

An embodiment of a method and apparatus for synchronizing routing protocol information associated with a plurality of routing modules of a network element is disclosed herein. The method includes an operation for adding an additional routing module to a network element. The network element includes an existing routing module having an existing collection of routing protocol information associated therewith. In response to adding the additional routing module to the network element, an operation is performed for imparting the existing collection of routing protocol information upon the additional routing module. After updating the existing routing module with such new routing protocol information, an operation is performed for updating the additional routing module with such new routing protocol information.

RELATED APPLICATIONS

[0001] The present application claims priority to U.S. ProvisionalPatent Application No. 60/352,100, filed on Jan. 24, 2002, having thesame title as the present application. The present application isrelated to the United States patent application entitled METHOD ANDAPPARATUS FOR PROVIDING REDUNDANT PROTOCOL PROCESSES IN A NETWORKELEMENT, having an attorney docket number of 1400.1375100, filed on thesame day as the present application and claiming priority to U.S.Provisional Patent Application No. 60/352,065, which was also filed onJan. 24, 2002. The present application is also related to the UnitedStates patent application entitled METHOD AND APPARATUS FORSYNCHRONIZING REDUNDANT COMMUNICATION TASKS, having an attorney docketnumber of 1400.1375130, filed on the same day as the present applicationand claiming priority to U.S. Provisional Patent Application No.60/351,683, which was also filed on Jan. 24, 2002.

FIELD OF THE DISCLOSURE

[0002] The present invention relates generally to networkcommunications, and more particularly to synchronization of redundantcommunication tasks.

BACKGROUND

[0003] Data communication protocols serve to facilitate transmission andreception of data across communication networks. For example,transmission control protocol (TCP), Internet protocol (IP), bordergateway protocol (BGP), asynchronous transfer mode (ATM), and variousother protocols facilitate communication of data between two or morelocations in a communication network. Through the use of such protocols,communication of data across a plurality of communication networks maybe facilitated, even though two or more of the networks comprisedifferent operating systems and architectures.

[0004] The Open Systems Interconnect (OSI) Reference Model developed bythe International Standards Organization (ISO) is generally used todescribe the structure and function of data communications. The OSIReference Model encompasses seven layers, often referred to as a stackor protocol stack, which define the functions of data communicationsprotocols. The protocol stack comprises a physical layer, a data linklayer, a network layer, a transport layer, a session layer, apresentation layer, and an application layer. A layer does not define asingle protocol, but rather a data communications function that may beperformed by any number of protocols suitable to the function of thatlayer. For example, a file transfer protocol and an electronic mailprotocol provide user services, and are thus part of the applicationlayer. Every protocol communicates with its peer, which is astandardized implementation of the identical protocol in the equivalentlayer on a remote system. For example, a local electronic mail protocolis the peer of a remote electronic mail protocol. As another example,BGP on a local router exchanges routing information with BGP on aneighboring router.

[0005] Applications, such as BGP, which require a transport protocol toprovide reliable data delivery, often use TCP because TCP verifies thatdata is delivered across a network (between separate end systems)accurately and in the proper sequence. TCP provides reliability with amechanism referred to as Positive Acknowledgement with Retransmission(PAR). In simplest terms, a system with PAR re-transmits the data forwhich it has not received an acknowledgement message from a far-endnode. Information is communicated between cooperating TCP modules insegments. A segment is a datagram containing a TCP header and perhapsdata. The TCP header contains sequence numbers. Control information,called a handshake, is exchanged between the two endpoints to establisha dialogue before data is transmitted.

[0006] As previously discussed, border gateway protocol (BGP) typicallyruns over TCP (e.g., port 179). BGP version 4 (BGP4) is the current defacto exterior routing protocol for inter-domain (autonomous systems)routing. BGP is a protocol used to advertise routes between networks ofrouters, e.g., between a Service Provider's network and a Carrier'snetwork. Routers at the edges of these networks exchange BGP messages,which could affect hundreds of thousands of routes. If the BGP processat one of these edge routers terminates (e.g., because of a restart,hardware failure, software upgrade, etc.), service on the routes betweenthe networks is usually affected. The termination also causes additionalBGP messages to be exchanged between other edge routers to updateinformation about available routes. Consequently, the terminationresults in a period of route instability and unavailability of theaffected router, which consequences are desirable to avoid. Furthermore,the termination will often result in a flood of re-routing messagesbeing sent into the network, thus adversely affecting performance of thenetwork.

[0007] A conventional BGP redundancy technique for addressing BGPprocess failures involves configuring two or more routers from differentvendors in parallel. The objective of such a technique is to reduce thepotential for BGP process failures by relying on the assumption that oneof the routers will survive at least some of the time a particular setof circumstances that might lead to failure of another router. Forexample, at least one of the routers would ideally exhibit immunity tofailure such as those that might be caused by an offending message, ahardware fault, or a software fault. That is, it is assumed that routersfrom different vendors are susceptible to different types of failures.This type of conventional BGP redundancy technique is generallyexpensive due to the inherent cost of the multiple routers and becauseusing equipment from multiple vendors causes additional operation,support, network management, and training costs. Additionally, this typeof conventional BGP redundancy technique requires additional BGPmessages to be exchanged to move the routes onto the tandem router, thusincreasing cost, complexity, and network traffic. The attached routersstill notice that the first router has disappeared and then route aroundit. Accordingly, it is desirable to avoid the disadvantages associatedwith such a conventional BGP redundancy technique.

[0008] A graceful restart mechanism for a router is another conventionaltechnique for addressing BGP process failures. Such a graceful restartmechanism is proposed in an Internet Engineering Task Force (IETF) draftentitled “Graceful Restart Mechanism for BGP”. In this proposal, arouter has the capability of preserving its forwarding state (routes)over a BGP restart, the ability to notify its peer routers of thiscapability and the ability to notify its peer routers of an estimatedtime for restart completion before it initiates such a restart. Upondetecting that the BGP process of the router has terminated (i.e., afailed router) and in response to receiving a correspondingnotification, the peer routers do not send new best routes toaccommodate for the failed router unless it fails to restart within thespecified time limit.

[0009] Such a graceful restart mechanism requires that the peer routersbe able to interpret and respond to the restart notification.Additionally, while the failed router is restarting it cannot processrouting updates that would normally be received. Consequently, itbecomes out of date during the period of unavailability, which isfollowed by a burst of updates once back in service These updates causeincreased “churn” in the routing tables of other routers, which affectsperformance of the network and should therefore be avoided. Even worse,routing loops or “blackholes” may form in this period of unavailability.Such “blackholes” occur when a route is advertised as available, butwhen the corresponding router is not actually configured to support sucha route, resulting in loss of packets intended to be communicated overthat route. Furthermore, the router may not actually be coming back intoservice. Also, since a graceful restart mechanism allows the specifiedtime limit for routers to be restarted, waiting that amount of time canincrease the time it takes to detect a failure and route around thefailed router. Additionally, implementation of such a grateful restartmechanism requires protocol extensions to BGP to which all routers awareof the failure must adhere in order to support the graceful restartmechanism. Accordingly, it is desirable to avoid the disadvantagesassociated with a graceful restart mechanism.

[0010] Therefore, facilitating synchronization of protocol tasks andrelated information on redundant routing modules of a network element ina manner that enables limitations associated with conventionalredundancy techniques to be overcome is useful.

BRIEF DESCRIPTION OF THE DRAWINGS

[0011]FIG. 1 is flow chart view depicting a method for synchronizing TCPtasks running on redundant routing modules of a network element inaccordance with an embodiment of the disclosures made herein.

[0012]FIG. 2 is a flow chart view depicting a method for facilitating anactivity switch in accordance with an embodiment of the disclosures madeherein.

[0013]FIG. 3 is a flow chart view depicting a method for synchronizingrouting protocol information associated with a plurality of routingmodules of a network element in accordance with an embodiment of thedisclosures herein.

[0014]FIG. 4 is a block diagram view depicting a network element 400capable of carrying out methods in accordance with embodiments of thedisclosures made herein.

DETAILED DESCRIPTION OF THE FIGURES

[0015] An embodiment of a method and apparatus for synchronizing routingprotocol information associated with a plurality of routing modules of anetwork element is disclosed herein. The method includes an operationfor adding an additional routing module to a network element. Thenetwork element includes an existing routing module having an existingcollection of routing protocol information associated therewith. Inresponse to adding the additional routing module to the network element,an operation is performed for imparting the existing collection ofrouting protocol information upon the additional routing module. Afterupdating the existing routing module with such new routing protocolinformation, an operation is performed for updating the additionalrouting module with such new routing protocol information.

[0016] The disclosures made herein pertain to various aspects offacilitating synchronization of redundant routing modules in a networkelement. In accordance with embodiments of the disclosures made herein,lower layer protocol (e.g., Transmission Control Protocol (TCP)) andhigher layer protocol (e.g., Border Gateway Protocol (BGP)) tasks of afirst routing module are synchronized with respective lower layerprotocol (e.g., TCP) and higher layer protocol (e.g., BGP) tasks of asecond routing module. The first routing module and the second routingmodule are redundant routing modules within a network element. Protocolinformation (e.g., TCP packets, BGP packets, etc) that is processed onthe first routing module (e.g., an active one of a plurality ofredundant routing modules) is similarly processed on the second routingmodule (e.g., an inactive one of the plurality of redundant routingmodules). Accordingly, such a network element in accordance with anembodiment of the disclosures made herein advantageously comprisesredundant, synchronized routing modules that are capable of supportingcarrier-grade quality of service over networks of various communicationprotocols (e.g., Internet Protocol, etc). A lower layer protocol (e.g.,TCP) packet (which may be referred to as a segment) is not necessarilycongruent with a higher layer protocol (e.g., BGP) packet. For example,it is not necessarily true that a TCP packet contains a BGP packet. Forexample, say a node is transmitting two BGP packets A and B and eachpacket includes 1000 bytes. A TCP task will most likely transmitportions of these BGP packets in separate TCP packets. For example, afirst TCP segment may contain 512 bytes of data (the first 512 bytes ofBGP packet A), a second TCP segment may contain 512 bytes of data (theremaining 488 bytes of BGP packet A, together with the first 24 bytes ofBGP packet B), a third TCP segment may contain 512 bytes of data (thenext 512 bytes of BGP packet B), and finally a fourth TCP segment maycontain 464 bytes of data (the remaining 464 bytes of BGP packet B). Theforegoing is merely an example, and other relationships between lowerlayer protocol packets and higher layer protocol packets are entirelypossible.

[0017] Embodiments of the disclosures made herein are capable ofenabling redundant lower layer protocol tasks (e.g., TCP tasks) andhigher layer protocol tasks (e.g., BGP tasks), thus allowing for anactivity switch without adversely affecting service. In suchembodiments, when an activity switch is implemented, disruption ofservice on routes distributed by such higher layer and lower layerprotocols is limited, if not eliminated. For example, after such anactivity switch, a newly active routing module (i.e., previously theinactive routing module) processes routing updates that would normallybe received by a newly inactive routing module (i.e., previously theactive routing module). Furthermore, the newly active routing moduledoes not become out of date with respect to routing informationmaintained on other network elements. In this manner, the network is notburdened by a burst of updates in response to the activity switch.Limiting the burden of such a burst of updates eliminates “churn” in therouting tables of network elements, thus improving performance of thenetwork. Significantly, service of existing routes is maintained, andchange to existing routes, deletion of routes, and addition of routescan continue uninterrupted; the switchover is transparent to neighboringrouters. By being transparent to neighboring routers, a techniquedisclosed herein need not require cooperation of neighboring routers toenable an activity switch. Accordingly, neighboring routers need not bemade aware of such an activity switch, nor do they need to supportprotocol extensions to enable such an activity switch.

[0018] Such embodiments are advantageous in that an offending packet ofinformation that results in failure of a higher layer protocol task of afirst routing module does not readily result in failure of the samehigher layer protocol task of a second routing module that is redundantand synchronized with respect to the first routing module. Oneembodiment of a technique for limiting the potential for failure of thesecond routing module from the offending packet of information is tomaintain a higher layer protocol task (e.g., a BGP task) of an inactiveone of a plurality of synchronized routing modules (i.e., an inactiverouting module) at least one higher layer protocol packet (e.g., a BGPpacket) behind the same higher layer protocol task of an active one ofthe plurality of synchronized routing modules (i.e., the active routingmodule). In this manner, the offending packet is recognized as suchprior to being processed by the higher layer protocol task of theinactive routing module, thereby avoiding the failure of the higherlayer protocol task of the inactive routing module that would otherwiseresult.

[0019] Another advantageous aspect of embodiments of the disclosuresmade herein is that processing power of a network element is notadversely affected. Specifically, synchronization and redundancy inaccordance with embodiments of the disclosures made herein arefacilitated in an efficient and effective manner. Accordingly, asignificant majority of processing power of the network element isavailable for performing primary tasks of the network element (e.g.,switching, routing, etc).

[0020] Still another advantageous aspect of embodiments of thedisclosures made herein is that such embodiments are less costly toimplement and maintain than conventional solutions. Such embodiments donot require redundant network elements, but rather redundant routingmodules within a particular network element. In some embodiments, theredundant routing modules are implemented identically, thus reducingcost. For example, similar software may be executed within each of theredundant routing modules. In other embodiments, differently-implementedredundant routing modules may be used.

[0021] It should be understood that embodiments of the present inventionmay be practiced with a variety of higher layer protocols. While BGPpackets are mentioned in many places herein, it should be understoodthat routes can also arrive from other protocols (e.g., Open ShortestPath First (OSPF)), or due to configuration changes (e.g., staticroutes). Not only are routes kept is sync between an active routingmodule and an inactive routing module, but so is configuration. Theconfiguration can also change on-the-fly (e.g., a BGP peer may be addedor removed at any time). It should be noted that a higher layer protocolmay be used for advertising routes, but may also be used for withdrawingroutes (e.g., a BGP packet may also specify a route to withdraw).

[0022] Turning now to the figures, a method 100 for synchronizing TCPtasks running on redundant routing modules of a network element inaccordance with an embodiment of the disclosures made herein is depictedin FIG. 1. The method is performed by a network element preferablycomprising a line card 134, an active routing module 136, and aninactive routing module 138. Various steps of the method are illustratedas being performed by the line card 134, by the active routing module136, and by the inactive routing module 138. The method 100 begins at anoperation 102 where a line card of a network element forwards a ProtocolData Unit (PDU), or a copy thereof, for reception by an active routingmodule and an inactive routing module of the network element. AnInternet Protocol routing module is an example of both the active andthe inactive routing modules. An apparatus capable of providing routingfunctionality (i.e., a router) is an example of the network element. Inother embodiments, the network element need not be implemented on arouter, but may by implemented on one or more other network devices. Asan example, for embodiments wherein the higher layer protocol packetsare Multi-Protocol Label Switching (MPLS) packets, the network elementmay be so implemented. Accordingly, the active routing module andinactive routing module may be considered more generically to be simplyan active module and an inactive module. The active routing moduleperforms an operation 104 for receiving the PDU while the inactiverouting module effectively ignores (e.g., receives but not process) thePDU in operation 140.

[0023] After receiving the PDU, the active routing module performs anoperation 106 for extracting a TCP packet encapsulated within the PDU.The TCP packet extracted from the PDU is hereinafter referred to as theinbound TCP packet. The active routing module TCP task performs anoperation 110 for receiving the first copy of the inbound TCP packet.

[0024] After the active routing module receives the first copy of theinbound TCP packet, the active routing module TCP task performs anoperation 114 for storing the first copy of the inbound TCP packet in areceive queue associated with the active routing module TCP task. Afteroperation 114, the active routing module performs operation 142 to makea determination as to whether or not the inbound TCP packet should beforwarded to the inactive routing module 138. If it is determined thatthe inbound TCP packet should not be forwarded, the process continues tooperation 144, where the inbound TCP packet is not forwarded. If it isdetermined that the inbound TCP packet should be forwarded, the processcontinues to operations 108 and 120. In operation 108, the activerouting module forwards a first copy of the inbound TCP packet forreception by a TCP task of the active routing module (i.e., the activerouting module TCP task) and a second copy of the inbound TCP packet forreception by a TCP task of the inactive routing module (i.e., theinactive routing module TCP task). In at least one embodiment, operation114 is performed before operation 108, while, in at least oneembodiment, operation 108 is performed before operation 114. It isimportant to note that the active routing module processes the incomingTCP packet and then, if appropriate, forwards the incoming TCP packet(along with other information) to the inactive routing module. Someincoming TCP packets, for example, acknowledgements that contain nodata, need not be forwarded to the inactive routing module. The inactiverouting module TCP task performs an operation 112 for receiving thesecond copy of the inbound TCP packet. Similarly, after the inactiverouting module receives the second copy of the inbound TCP packet, theinactive routing module TCP task performs an operation 116 for storingthe second copy of the inbound TCP packet in a receive queue associatedwith the inactive routing module TCP task. The operation 116 for storingthe second copy of the inbound TCP packet in a receive queue associatedwith the inactive routing module TCP task includes initially storingsecond copy of the inbound TCP packet in a pending portion of thereceive queue associated with the inactive routing module TCP task(i.e., the pending portion of the inactive routing module receivequeue).

[0025] In operation 120, a BGP task of the active routing module (i.e.,the active routing module BGP task) facilitates recordation of the peernetwork element from which the inbound TCP packet was received. Asdiscussed below in greater detail in reference to FIG. 2, such a recordof the peer network element from which the inbound TCP packet wasreceived is used for facilitating an activity switch from the activerouting module to the inactive routing module if a failure occurs whileprocessing the BGP packet. After operation 120, the active routingmodule BGP task performs an operation 118 for processing the BGPmessage.

[0026] After operation 118, an operation 121 is performed fordetermining whether processing of the BGP message contained in the firstcopy of the inbound TCP packet is performed successfully. When theoperation 118 for processing a BGP message contained in the first copyof the inbound TCP packet is successfully performed, the inactiverouting module TCP task performs an operation 122 for storing the secondcopy of the inbound TCP packet in a ready portion of the receive queueassociated with the inactive routing module TCP task (i.e., the readyportion of the inactive module receive queue). In at least oneembodiment of the operation 122, the operation 122 for storing thesecond copy of the inbound TCP packet in the ready portion of theinactive routing module receive queue includes forwarding the secondcopy of the inbound TCP packet from the pending portion to the readyportion of the inactive routing module receive queue. Upon adetermination in operation 121 that processing of the BGP messagecontained in the first copy of the inbound TCP packet was performedsuccessfully, the second copy of the inbound TCP packet can beimmediately moved from the pending portion to the ready portion of theinactive routing module receive queue, but, in at least one embodiment,it is advantageous to cause such action to occur at a later time forperformance reasons. For example, an instruction for the inactiverouting module to perform operation 122 may be included within otherinformation destined for the inactive routing module to avoid the needto send the instruction separately and to minimize the amount ofinformation being sent to the inactive routing module and the amount ofprocessing required by the inactive routing module.

[0027] After, and only after, the second copy of the inbound TCP packetis stored in the ready portion of the inactive routing module receivequeue, the inactive routing module BGP task performs an operation 124for processing the BGP message contained in the second copy of theinbound TCP packet. The active routing module performs an operation 126for issuing an acknowledgement message for designating that the inboundTCP packet has been received. In at least one embodiment, operation 126is performed after operation 122, while in at least one otherembodiment, operation 126 is performed before operation 122, as long asit is performed after operation 116. The operation for initially storingthe second copy of the inbound TCP packet in the pending portion of theinactive routing module receive queue and then in the ready portion ofthe inactive routing module receive queue enables the second copy of theinbound TCP packet to remain unprocessed by the inactive routing moduleBGP task until the contents of the inbound TCP packet is determined tobe non-offensive (e.g., not causing a BGP task failure) via the activerouting module BGP task successfully processing the first copy of theinbound TCP packet. In this manner, the inactive routing module BGP taskprocesses a particular BGP messages after the active routing module BGPtask processes the particular BGP message.

[0028] In at least one embodiment of the active routing module andinactive routing module BGP tasks, such BGP tasks are precluded fromreceiving partial TCP packets from the TCP task. Such partial packetsmay contain partial BGP messages therein, potentially causingsynchronization problems when an activity switch is implemented. It iscontemplated herein that a TCP task may be configured such it precludesan associated BGP task from recognizing that information of a packet isbeing received until the information comprises a full packet. A socketoption exists for enabling such functionality.

[0029] By enabling the BGP message contained in the second copy of theinbound TCP packet to be processed only after second copy of the TCPpacket is stored in the ready portion of the inactive routing modulereceive queue, it is assured that such processing of the BGP messagecontained in the second copy of the TCP packet will take place onlyafter the BGP message contained in the first copy of the TCP packet issuccessfully processed by the active routing module BGP task. It shouldbe understood that the BGP messages contained in the first copy and thesecond copy of the inbound TCP packet are essentially identical (i.e.,the same BGP message). Accordingly, the potential for failure of theactive and inactive routing module BGP tasks resulting from the same BGPmessage is substantially reduced, if not eliminated.

[0030] Issuing the acknowledgement message for designating that theinbound TCP packet has been received (e.g., operation 126) only afterthe second copy of the TCP packet has been stored in the inactiverouting module receive queue pending portion (e.g., operation 116)assures that the inactive routing module will not fail to receive anyTCP packets, even in the event of an activity switch. Thus, in the eventthat an activity switch does not occur, the BGP message contained in thefirst copy of the TCP packet is successfully processed by the activerouting module BGP task. Moreover, upon an activity switch, the TCPcommunication with the peer network element that transmitted a TCPpacket including an offending BGP message is terminated. Thisoperational sequence ensures that a TCP packet including an offendingBGP message is not processed after such message results in failure ofthe active routing module BGP task. Thus, redundancy robustness isenhanced.

[0031] Turning now to update messages being transmitted from the networkelement for reception by its peer network elements, it will beappreciated that redundancy can also be provided where such outboundupdate messages are concerned. For example, in response to receiving aBGP message designating a new route, a route update message istransmitted from the network element for reception by one or more of itspeer network elements for notifying such peer network elements of thenew route.

[0032] Accordingly, in response to the active routing module receivingsuch types of BGP messages that necessitate an outbound update messageor receiving a route from another protocol (e.g., OSPF or ISIS), or inresponse to an internal event (e.g., a configuration change, such as,for example, adding a static route) for which an update message shouldbe generated, an operation 128, FIG. 1, is performed for storing a firstcopy of an outbound BGP packet encapsulated within one or more TCPpackets in a transmit queue of the active routing module (i.e., theactive routing module transmit queue). Operation 128 after operation118. Also, in response to the active routing module receiving such typesof BGP messages that involve an outbound update message, an operation130 is performed for storing a second copy of the outbound BGP packetencapsulated within one or more TCP packets in a transmit queue of theinactive routing module (i.e., the inactive routing module transmitqueue). Similar to receive queue functionality as disclosed herein, inat least one embodiment of the active routing module and inactiverouting module transmit queues, such transmit queues are precluded fromstoring partial BGP packets. An operation 132 is performed forforwarding the first copy of the outbound BGP packet encapsulated withinone or more TCP packets from the active routing module transmit queuefor reception by one or more peer network elements only after the secondcopy of the outbound BGP packet encapsulated within one or more TCPpackets is stored in the inactive routing module transmit queue. In thismanner, retransmission and packet sequencing functionality aremaintained after an activity switch from the active routing module tothe inactive routing module.

[0033] Referring back to the operation 121, this operation is alsocapable of determining whether processing of the BGP message containedin the first copy of the inbound TCP packet is not successfullyperformed. In response to the BGP message contained in the first copy ofthe inbound TCP packet being not being processed successfully by theactive routing module BGP task, an activity switch is facilitated, andthe process is directed to an entry point “A.” The activity switchtransfers on-line operations of the network element from the previouslyactive routing module (now the inactive routing module) to a newlyactive routing module (previously the inactive routing module).

[0034]FIG. 2 depicts a method 200 for facilitating an activity switch inaccordance with an embodiment of the disclosures made herein. The method200 pertains to an activity switch resulting from an offending inboundTCP packet. A TCP packet including an offending BGP message is oneexample of the offending TCP packet.

[0035] At an entry point “A” corresponding to processing of an errantBGP packet of an inbound TCP packet (i.e., one for which satisfactoryerror handling has not otherwise been provided), the method 200 beginswith an error handling routine invoked by a system controller implementsan operation 202 for identifying a peer network element from which theoffending inbound BGP packet was received. One embodiment of identifyingthe identified peer network element includes reading/accessing a recordgenerated in response to the operation 120, FIG. 1, for facilitatingrecordation of the peer network from which the offending inbound TCPpacket was received. It should be noted that, in at least oneembodiment, when recording the network element from which a packet hasbeen received, the peer network element of the BGP packet is recorded,not that of the TCP segment. It is possible that the BGP peer networkelement and TCP peer network element may be different. BGP may have asession with a neighbor that requires multiple TCP hops to reach. Assuch, the identified peer network element is identified with respect tothe higher layer protocol packet (e.g., the BGP packet).

[0036] The system controller may, for example, be a control element,such as a processor, coupled to the network element or incorporatedwithin the network element. For example, the system controller may beimplemented as a process that reads the record to determine the peernetwork element from which the packet was received and to initiate thetermination of the associated BGP session on the inactive routingmodule. In at least one embodiment, this process is contained within theactive routing module. When required to terminate a session, the activerouting module (e.g., the system controller contained within the activerouting module) communicates with the inactive routing module as towhich peering session to terminate.

[0037] After identifying the peer network element from which theoffending inbound BGP packet was received (i.e., the identified peernetwork element), the error handling routing performs an operation 204for initiating termination of a BGP session associated with theidentified peer network element and an operation 206 for initiatingtermination of a TCP session associated with the identified peer networkelement. Since, in at least one embodiment, initiating termination of aBGP session will inherently initiate termination of a TCP session,operations 204 and 206 may optionally be performed as a singleoperation. Likewise, such a single operation may result in performanceof operation 208, which may inherently result in performance ofoperation 210. After initiating termination of the BGP and TCP sessionsassociated with the identified peer network element, the newly activerouting module TCP task performs an operation 210 for terminating theTCP session associated with the identified peer network element, and thesoon-to-be-newly active routing module BGP performs an operation 208 forterminating the BGP session associated with the identified peer networkelement. In response to facilitating termination of the TCP sessionassociated with the identified peer network element, the newly activerouting module TCP task performs an operation 212 for purging theoffending inbound TCP packet from the receive queue of the newly activerouting module. The actual switching of functional operations isfacilitated after the TCP and BGP sessions are terminated and theoffending inbound TCP packet is purged from the receive queue of thenewly active routing module. Because the TCP session has beenterminated, even if the offending inbound TCP packet has not beenacknowledged, it will not be re-sent, thereby avoiding a failure of thenewly active routing module.

[0038] As an additional precaution, TCP and BGP task sessions with theidentified peer network element are re-established after an operation214 is performed for restarting the newly inactive module and untilafter an operation 216 is performed for synchronizing existingrouting-related information of the newly inactive routing module withthe newly active routing module. Such routing-related information mayinclude information stored in a routing information database, as well asother information, such as configuration information (e.g., staticconfiguration information) and state information (e.g., dynamic stateinformation). In response to synchronizing such existing routing-relatedinformation, the newly active routing module BGP task performs anoperation 220 for re-establishing a BGP session with the identified peernetwork element. To re-establish a BGP session in accordance withoperation 220, the newly active routing module TCP task performs anoperation 218 for re-establishing a TCP session with the identified peernetwork element. In this manner, risk associated with re-establishingsuch task sessions with the identified network element without aredundant routing module being in place are reduced, if not eliminated.Optionally, in at least one embodiment, BGP and TCP task sessions aremaintained with other peer network elements besides the identified peernetwork element.

[0039]FIG. 3 depicts a method 300 for synchronizing routing protocolinformation associated with a plurality of routing modules of a networkelement in accordance with an embodiment of the disclosures herein. Bysynchronizing such routing protocol information, redundancy inaccordance with the disclosures made herein may be implemented. Suchsynchronization contributes to enabling an activity switch from a firstrouting module of the network element to a second routing module of thenetwork element in an essentially transparent manner with respect topeer network elements.

[0040] The method 300 begins with an inactive routing module performingan operation 302 for receiving a copy of existing routing protocolinformation from an active routing module. The operation 302 isperformed in response to the inactive routing module being an additionalrouting module that is added to a network element including the activerouting module. Because the active routing module is an existing, in-userouting module of the network element, the active routing module hassuch existing routing protocol information associated therewith prior toaddition of the inactive routing module. For example, it may be the casethat at least a portion of the routing protocol information wasdynamically established in the existing routing module over a period oftime prior to the addition of the additional routing module to thenetwork element. Examples of routing protocol information include TCPrelated state information, BGP configuration, BGP routing tables, androute state information (e.g., designation that a route has beenadvertised to peer network elements).

[0041] In response to the inactive routing module receiving suchexisting routing information, an operation 304 is performed for updatinginactive routing module records associated with such routing protocolinformation. An embodiment for updating such inactive routing modulerecords associated with such existing routing protocol informationincludes updating a routing information database of the inactive routingmodule. In one embodiment of the inactive routing module, the inactiverouting module does not include any existing routing protocolinformation (e.g., the inactive routing module is a new routing modulebeing put into service). In another embodiment of the inactive routingmodule, the inactive routing module includes existing routing protocolinformation that is being updated.

[0042] At some point in time after the inactive routing module is addedto the network element and during the normal course of operation of theactive routing module, the active routing module performs an operation306 for receiving a first copy of new routing protocol information(newly-received routing protocol information) from one or more peernetwork elements. In response to receiving such newly-received routingprotocol information, the active routing module performs an operation308 for updating active routing module records associated with suchnewly-received routing protocol information, an operation 312 forforwarding a second copy of such newly-received routing protocolinformation for reception by the inactive routing module, and anoperation 310 for acknowledging receipt of such newly received routingprotocol information. Thus, acknowledgement is provided to the one ormore peer network elements from which the new routing protocolinformation was received after a copy of such new routing protocolinformation (or the portion thereof for which the acknowledgement isbeing provided) has been forwarded to the inactive routing module (i.e.,the additional routing module). After the active routing module forwardssuch newly-received routing protocol information for reception by theinactive routing module, the inactive routing protocol performs anoperation 314 for receiving such newly-received routing protocolinformation and an operation 316 for updating inactive routing modulerecords associated with such routing protocol information. It should benoted that operations 304 and 316 may be performed as separateoperations or combined into a single operation. A TCP packet including aBGP message is an example of such newly-received routing protocolinformation during the normal course of operation of the active routingmodule.

[0043] Referring now to FIG. 4, a network element 400 capable ofcarrying out methods in accordance with embodiments of the disclosuresmade herein is depicted. Specifically, the network element 400 iscapable of carrying out redundancy and synchronization functionality inaccordance with the disclosures made herein. For example, the networkelement 400 is capable of carrying out the methods disclosed herein(e.g., the methods 100, 200 and 300). An apparatus capable of providingrouting functionality (e.g., a router) is an example of the networkelement 400.

[0044] The network element 400 includes an active routing module 402(i.e., the first routing module), an inactive routing module 404 (i.e.,the second routing module), and a line card 405 connected between theactive and inactive routing modules (402, 404). The line cardfacilitates routing a respective copy of each inbound TCP packet (e.g.,via forwarding of corresponding Protocol Data Units (PDUs)). However,the TCP task of the inactive routing module 402 ignores such TCP packets(e.g., does not process the PDUs) while the TCP task of the activerouting module 402 processes such TCP packets.

[0045] The active routing module 402 and the inactive routing module 404are capable of facilitating redundant functionality in according withthe disclosures made herein. The active routing module 402 and theinactive routing module 404 each include respective TCP tasks (406,408), respective BGP tasks (410, 412) and respective routing informationdatabases (414, 416). The TCP tasks (406, 408) are each examples oflower layer protocol tasks. The BGP tasks (410, 412) are each examplesof higher layer protocol tasks. It is contemplated herein that BGP tasks(410, 412) may be substituted with other protocols that use TCP toexchange messages (e.g., multi-protocol label switching (MPLS)).

[0046] The TCP task 406 of the active routing module includes a receivequeue 418 and a transmit queue 420. The TCP task 408 of the inactiverouting module 404 includes a receive queue 422 and a transmit queue424. The receive queue 422 includes a pending portion 426 and a readyportion 428. The pending portion 426 and the ready portion 428 of theinactive routing module receive queue 422 facilitate functionality asdepicted in FIG. 1. Specifically, the pending portion 426 and the readyportion 428 of the inactive routing module receive queue 422 enables aparticular copy of a TCP packet to remain unprocessed by the inactiverouting module BGP task 412 until the contents of such TCP packet isdetermined to be non-offensive (e.g., not causing a BGP task failure) bythe BGP task 410 of the active routing module 402. This, the inactiverouting module BGP task 412 processes a particular BGP message after theactive routing module BGP task 410 processes the particular BGP message.

[0047] In some embodiments of the inactive routing module, the inactiverouting module 404 does not receive flow control updates from the activerouting module 402. Thus, it is theoretically possible for the inactiverouting module receive queue 422 to overflow. To reduce thispossibility, the inactive routing module receive queue 422 is preferablymuch larger than the active routing module receive queue 418 in suchembodiments. However, it should be understood that the inactive routingmodule 404 does less work than the active routing module 402 (e.g., theflooding responsibilities are greatly reduced, the TCP/IP stack is nottransmitting data, etc.). Accordingly, there should not be a steadystate possibility where the inactive routing module receive queue 422continues to grow without limit.

[0048] It should be understood that the active routing module 402 iscapable of supporting functionality disclosed herein in association withthe inactive routing module 404 and the inactive routing module 404 iscapable of supporting functionality disclosed herein in association withthe active routing module 402. Accordingly, in the event of an activityswitch in accordance with the disclosures made herein, the activerouting module 402 (i.e., the newly inactive routing module) providesfunctionality previously provided by the inactive routing module 404(i.e., the newly active routing module) and the inactive routing module404 provides functionality previously provided by the active routingmodule 402. For example, after an activity switch, the active routingmodule 402 provides functionality associated with the pending queue 426and ready queue 428 of the inactive routing module 404.

[0049] In accordance with at least one embodiment of the disclosuresmade herein, the BGP tasks of the active routing module 402 and theinactive routing module 404 do not queue any transmit data on a per-peer(i.e., per socket) basis. One reason that the BGP tasks no longer queueon a per-peer basis is because data queued in the BGP task would not beguaranteed of delivery after an activity switch. Another reason is thatsynchronization of lists of routes which need to be advertised orwithdrawn would be excessively intensive if BGP task transmit queuesneeded searching.

[0050] It is contemplated herein that the active routing module transmitqueue 420 is enlarged in order to enable omission of a transmit queue ofthe active routing module BGP task. That is, the transmit queue 420 ofthe active routing module TCP task 406 needs to be large enough toensure that transmissions continue between successive periods ofprocessing of advertised or withdrawn routes.

[0051] Because the BGP tasks (410, 412) of the active and inactiverouting modules (402, 404) cannot queue any transmit data, an operationfor transmitting data to the active routing module TCP task 406 mustsucceed. Otherwise, the active routing module BGP task 410 would have toqueue such transmit data, which it preferably does not do. To ensurethat the operation for transmitting data to the active routing moduleTCP task 406 succeeds, the active routing module BGP task 410 firstensures that sufficient space exists in the transmit queue 420associated with the active routing module TCP task 406. In oneembodiment, ensuring that such sufficient space exists is accomplishedvia a read in shared memory. To this end, the TCP task 406 of the activerouting module 402 maintains a table of free space in the active routingmodule transmit queue 420. However, in other embodiments, othertechniques may be used for ensuring that such sufficient space exists.

[0052] Referring now to data processor programs in accordance with anembodiment of the disclosures made herein, a data processor programcontrols at least a portion of the operations associated withsynchronizing higher layer protocol tasks (e.g., BGP) and lower layerprotocol tasks (e.g., TCP) running on redundant routing modules of anetwork element. In this manner, the data processor program controls atleast a portion of the operations necessary to facilitate routing modulesynchronization functionality in a manner consistent with thedisclosures made herein. The term data processor program is definedherein to refer to computer software, data processor algorithms or anyother type of instruction code capable of controlling operationsassociated with a data processor. A microprocessor, microcontroller,microcomputer, digital signal processor, state machine, logic circuitry,and/or any device that manipulates digital information based onoperational instruction, or in a predefined manner are examples of adata processor.

[0053] A data processor program in accordance with an embodiment of thedisclosures made herein is processible by a data processor of an activeand/or inactive routing module of a network element. A copy of the dataprocessor program may be resident on each of the routing elements in anetwork element. Furthermore, each copy of the data processor programmay be accessible by a data processor of the respective routing modulefrom a memory apparatus of the respective routing module (e.g., RAM,ROM, virtual memory, hard drive memory, etc.) or from a peripheralapparatus such as a diskette, a compact disk, an external data storagedevice and the like.

[0054] A data processor program accessible from an apparatus by a dataprocessor is defined herein as a data processor program product. It iscontemplated herein that the data processor program product may comprisemore than one data processor programs each accessible from respectiveapparatuses. It is further contemplated herein that each one of aplurality of data processor programs may be accessed by a differentrespective one of a plurality of data processors. For example, a firstdata processor and a second data processor (e.g., of a leaf node and aroot node), respectively, may access a first data processor program anda second data processor program, respectively, from a first apparatusand a second apparatus (e.g., a first memory device and a second memorydevice), respectively.

[0055] In the preceding detailed description, reference has been made tothe accompanying drawings that form a part hereof, and in which areshown by way of illustration specific embodiments in which the inventionmay be practiced. These embodiments have been described in sufficientdetail to enable those skilled in the art to practice the invention, andit is to be understood that other embodiments may be utilized and thatlogical, mechanical, chemical and electrical changes may be made withoutdeparting from the spirit or scope of the invention. To avoid detail notnecessary to enable those skilled in the art to practice the invention,the description omits certain information known to those of skill in theart. The preceding detailed description is, therefore, not to be takenin a limiting sense, and the scope of the present invention is definedonly by the appended claims.

What is claimed is:
 1. A method for facilitating routing protocol redundancy in a network element, comprising: adding an additional routing module to the network element, wherein the network element includes an existing routing module having an existing collection of routing protocol information associated therewith; and synchronizing a higher layer protocol task of the additional routing module with a higher layer protocol task of the existing routing module in response to adding the additional routing module.
 2. The method of claim 1 wherein synchronizing the higher layer protocol task of the additional routing module with the higher layer protocol task of the existing routing module includes: imparting the existing collection of routing protocol information upon the additional routing module in response to adding the additional routing module to the network element; updating the existing routing module with new routing protocol information; and updating the additional routing module with the new routing protocol information.
 3. The method of claim 2 wherein: updating the additional routing module with the new routing protocol information is performed after a higher layer protocol task of the existing routing module processes the new routing protocol information; and the new routing protocol information is a higher layer protocol packet.
 4. The method of claim 3 wherein: the higher layer protocol packet specifies a route to advertise; and processing the higher layer protocol packet includes advertising the route.
 5. The method of claim 2, further comprising: forwarding at least a portion of the new routing protocol information to the additional routing module; and acknowledging receipt of the at least the portion of the new routing protocol information, wherein the step of forwarding the at least the portion of the new routing protocol information to the additional routing module occurs prior to the step of acknowledging receipt of the at least the portion of the new routing protocol information.
 6. The method of claim 2 wherein the new routing protocol information includes at least one of a BGP message, TCP task-related state information, BGP routing tables and route state information.
 7. The method of claim 1 wherein the existing collection of routing protocol information includes at least one of BGP message, TCP task-related state information, BGP routing tables and route state information.
 8. The method of claim 1 wherein adding the additional routing module includes adding the additional routing module after the existing routing module has been in operation for a period of time and wherein at least a portion of the routing protocol information was dynamically established over the period of time.
 9. Apparatus for facilitating routing protocol redundancy in a network element, comprising: an existing routing module having an existing collection of routing protocol information associated therewith; and an additional routing module coupled to the existing routing module, wherein a higher layer protocol task of the additional routing module is synchronized with a higher layer protocol task of the existing (routing module in response to coupling the additional routing module to the existing routing module.
 10. The apparatus of claim 9 wherein synchronization of the higher layer protocol task of the additional routing module with the higher layer protocol task of the existing routing module includes imparting the existing collection of routing protocol information upon the additional routing module in response to adding the additional routing module to the network element, updating the existing routing module with the new routing protocol information, and updating the additional routing module with the new routing protocol information.
 11. The apparatus of claim 10 wherein updating of the additional routing module with the new routing protocol information is performed after a higher layer protocol task of the existing routing module processes the new routing protocol information and wherein the new routing protocol information is a higher layer protocol packet.
 12. The apparatus of claim 11 wherein the higher layer protocol packet specifies a route to advertise and processing the higher layer protocol packet includes advertising the route.
 13. The apparatus of claim 10 wherein the existing routing module forwards at least a portion of the new routing protocol information to the additional routing module prior to acknowledging receipt of the at least the portion of the new routing protocol information.
 14. The apparatus of claim 10 wherein the new routing protocol information includes at least one of a BGP message, TCP task-related state information, BGP routing tables and route state information.
 15. The apparatus of claim 9 wherein the existing collection of routing protocol information includes at least one of BGP message, TCP task-related state information, BGP routing tables and route state information.
 16. The apparatus of claim 9 wherein the additional routing module is added after the existing routing module has been in operation for a period of time and wherein at least a portion of the routing protocol information was dynamically established over the period of time. 